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Executive summary 


The Parent Leadership Institute (PLI) of Children’s Aid (CA), funded via a 2013 Investing in Innovation (i3) 
development grant operated between the 2014-15 through 2018-19 school years. Key goals of the PLI 
included: (1) improving the capacity of parents to effectively engage in the school community in support of 
their child and (2) increasing the capacity of school staff to create and support environments which are 
welcoming to and supportive of the active engagement of parents as key members of the school 
community. Through implementation of the PLI, CA expanded its partnership with six schools located in the 
South Bronx community of Morrisania, an area characterized by high levels of poverty, health disparities, 
and crime, and low levels of academic achievement and attainment among both children and adults. 


The focus of the PLI and its work on building the capacity of parents to be leaders and advocates for 
their child aligns with the process of stage setting as identified by Robinson and Harris (2014). The 
researchers identify stage setting as parent efforts to “construct and manage the social environment 
around their children in a way that creates conditions where success is possible (p. 200).” The PLI, with 
its focus on building parent capacity, has encouraged partner schools to think creatively about ways to 
authentically encourage parents to become more engaged in their child’s school and education. The 
project director stated, “what you’re trying to do is to get ways that maybe you can push outside of 
getting them [parents] in there for doughnuts for dads or muffins for moms. Getting beyond that to talk 
about the real, authentic ways of encouraging parents to get involved.” 


Staffing and Program Components 


At the heart of the PLI are the CA parent engagement coordinators (PECs) who collaborate with parents, 
school staff, staff from local community organizations, and other CA staff to develop and deliver 
programming that meets the needs of the parents and families served by participating schools. The 
coordinators are supported by a project director who manages PEC hiring, provides training to PECs and 
school staff, facilitates parent engagement opportunities with teachers, and works with PECs to 
coordinate campus parent engagement activities. The PEC serves as one of three core components of 
the PLI which also includes a separate space in the school for parents and the delivery of tailored 
services and supports to parents and school staff to improve parent skills and capacity to effectively 
engage in schools. Jointly, these three program components reflect the desired operation and potential 
impact of the PLI. 


Each of the PLI partner schools continued to operate as a New York community school during the school 
year with Children’s Aid serving as the lead community school partner for all but one school. Although 
six schools, including the four co-located schools, are included as part of the formal i3 grant, the schools 
are also co-located with three additional unfunded schools which the project director and PECs indicate 
are also included in all provided activities and services to the extent possible. 


Evaluation Goals and Methods 


Children’s Aid contracted with Policy Studies Associates in early 2014 to conduct an evaluation of the 
implementation and impact of the PLI in participating schools. The evaluation period included the 2014- 


15 through 2016-17 school years. Throughout the evaluation period, the evaluation explored the 
following proposition: 


Through the coordinated efforts of Parent Resource Centers, parent engagement coordinators, 
and adult education and leadership development activities, parents will develop home 
environments that promote learning and will forge stronger connections with their children’s 
schools, resulting in student achievement and attendance that exceed those of similar students 
enrolled in matched schools. 


Implementation Evaluation 


The primary goal of the final, Year 4 implementation evaluation was to document via interviews with 
representatives of key stakeholder groups and analyses of administrative data on parent participation in 
sponsored services, strategies for implementing PLI during the 2016-17 school year. Of interest was 
learning about ways that CA staff worked with school staff to integrate parent engagement activities 
into overall school operation. In late spring 2016-17 members of the evaluation team conducted 
individual interviews or focus groups with more than 30 stakeholders representing each of the 
participating partner schools, the New York Department of Education, a partner organization, and CA 
staff. Analyses of these interviews serve as the primary basis for this report. To supplement these 
analyses, the evaluation team also analyzed CA administrative data on the types of services and 
supports provided to parents. These data provide additional context to the evaluation and provide 
information on “fidelity of implementation” as required by i3. 


Impact Evaluation 


The final phase of the evaluation incorporated quantitative analyses of data on student academic 
progress and performance prior to and during PLI implementation. To isolate the potential impact of 
the PLI on student outcomes, the evaluation compared6 the performance of the six PLI schools in 
mathematics, English/language arts, and attendance with that of 18 similar non-participating schools 
New York City public schools. The primary analytic method was an interrupted time-series model which 
looks for shifts or changes in key outcome areas after implementation of a new intervention or program 
compared with trends prior to the change in practice. 


Findings 


For the 2016-17 school year, CA achieved implementation targets in the areas if staffing, dedicated 
space, activity/workshop offerings, and parent participation in activities. Implementation targets for 
offering specialized services to individual parents and connecting individual parents with information on 
community resources were not met for the school year. In Year 4, PLI staff made increased efforts to 
work more collaboratively with school-based staff. However, despite these efforts, PECs continued to 
struggle to develop a shared vision for parent engagement grounded in parent capacity-building on 
nearly all campuses. Teacher participation in activities also varied in Year 4 with PECs reporting 
increased efforts to reach out to and build connections with classroom teachers around parent support 
for academics. 


Analyses of quantitative data on student academic performance and engagement yielded no statistically 
significant differences in student performance of state reading or mathematics assessments for PLI and 


comparison schools. Similarly, there was no statistically significant difference in attendance rates for 
students attending PLI and comparison schools. However, the raw difference of 3.5 percentage points 
between PLI and comparison schools provides potential evidence for a later impact in this area. PLI staff 
worked more consistently and directly with school staff on activities designed to improve school 
attendance rates than they worked with school staff on improving student academic performance. 


Conclusions and Recommendations 


The PLI continued to operate as initially planned for most key program components. The PLI was 
successful in meeting implementation targets for (1) PEC staffing, (2) the availability of a resource room, 
(3) the delivery of a diverse array of activities and workshops for parents, and (4) the number of parents 
participating in activities and workshops and achieving leadership status. Parent participation in 
activities and parents achieving leadership levels of participation increased annually between Years 2 
and 4. Reflecting both challenges in recording contacts with individual parents and a shift away from a 
case management focus, PECs did not meet implementation targets for the two indicators which 
centered on individualized services for parents. 


Over time PECs increased their efforts to work collaboratively with principals, teachers, and community 
school staff. These collaborative efforts are critical for supporting the integration of the PLI into overall 
school operations. Consistent outreach and collaboration in this area will likely help with the 
development of a shared vision for parent engagement. Given the importance of the development of 
partnerships with other members of the school community, CA may want to consider elevating its focus 
on this aspect of the work both when selecting new school partners and throughout PLI design and 
implementation. Interviews with CA and other school-based staff highlight the importance of 
understanding and adjusting to school contextual conditions for PLI success. In this light, developing 
and maintaining effective relationships with both parents and school staff should be at the forefront of 
PLI implementation and planning efforts. 


Although impact analyses did not find statistically significant effects on students’ educational outcomes, 
the evaluation’s findings suggest a potential for parent-centered programs to reduce chronic 
absenteeism among students. The PLI’s direct messaging about the importance of school attendance 
may have influenced parents’ efforts to ensure that their children attended school as often as possible. 
Further, as program staff hypothesized, the PLI may have indirectly supported improved attendance by 
helping parents and other caregivers improve their health and strengthen their connections with other 
parents. Additional investigation and analysis may be able to establish causal links between the PLI’s 
program components and improved attendance at school. 


Overview of the Children’s Aid Parent Leadership Institute 


The Parent Leadership Institute (PLI) of Children’s Aid (CA), funded via a 2013 Investing in Innovation (i3) 
development grant operated between the 2014-15 through 2018-19 school years. Key goals of the PLI 
included: (1) improving the capacity of parents to effectively engage in the school community in support of 
their child and (2) increasing the capacity of school staff to create and support environments which are 
welcoming to and supportive of the active engagement of parents as key members of the school 
community. Through implementation of the PLI, CA expanded its partnership with six schools located in the 
South Bronx community of Morrisania, an area characterized by high levels of poverty, health disparities, 
and crime, and low levels of academic achievement and attainment among both children and adults. 


National and local organizations have long advocated for the implementation of parent engagement 
activities by schools as a potential means of 


improving student academic performance and “ALL OF THE WORKSHOPS ARE PURPOSEFUL. 
engagement (Ishimaro et al., 2016). However, IT’S NOT A WORKSHOP JUST TO SAY, OKAY, WE 
although there is consensus that families play a NEED TO COLLECT NAMES. WE’RE ACTUALLY 
significant role in a student’s academic achievement DOING THIS TO SERVE OUR COMMUNITY.” 
and engagement (Cabrera& LaNasa, 2001; Jeynes Teacher from a PLI School 


2005; Roderick, Nagaoka, and Coca, 2011; Perna & 
Titus, 2005; Robinson & Harris, 2014), there are divergent opinions about what effective parent 
engagement looks like in practice and the role that school-based staff can play in fostering deeper levels 
of parent engagement (Ishimaru et al., 2016; Mapp & Kutner, 2013; Robinson & Harris, 2014). 


For each year of its operation, the PLI advocated for a model for parent engagement that was grounded 
in the premise that active, direct services and supports delivered to parents will enable parents or other 
participating to be better-equipped to support the positive development of their children. The focus on 
parents as recipients of services in the PLI framework broadens the scope of parent and family 
engagement efforts beyond that of traditional school-based activities which often center the children 
with parents primarily encouraging and supporting higher levels of performance for their child. In 
contrast, in recognition of the key roles that parents play in establishing productive learning 
environments for their children, the PLI actively centers parents as the focus of services and support. 


The focus of the PLI and its work on building the capacity of parents to be leaders and advocates for 
their child aligns with the process of stage setting as identified by Robinson and Harris (2014). The 
researchers identify stage setting as parent efforts to “construct and manage the social environment 
around their children in a way that creates conditions where success is possible (p. 200).” The PLI, with 
its focus on building parent capacity, has encouraged partner schools to think creatively about ways to 
authentically encourage parents to become more engaged in their child’s school and education. The 
project director stated, “what you’re trying to do is to get ways that maybe you can push outside of 
getting them [parents] in there for doughnuts for dads or muffins for moms. Getting beyond that to talk 
about the real, authentic ways of encouraging parents to get involved.” 


Staffing and Program Components 


At the heart of the PLI are the CA parent engagement coordinators (PECs) who collaborate with parents, 
school staff, staff from local community organizations, and other CA staff to develop and deliver 
programming that meets the needs of the parents and families served by participating schools. The 
coordinators are supported by a project director who manages PEC hiring, provides training to PECs and 
school staff, facilitates parent engagement opportunities with teachers, and works with PECs to 
coordinate campus parent engagement activities. The PEC serves as one of three core components of 
the PLI which also includes a separate space in the school for parents and the delivery of tailored 
services and supports to parents and school staff to improve parent skills and capacity to effectively 
engage in schools (Exhibit 1). Jointly, these three program components reflect the desired operation and 
potential impact of the PLI. 


Exhibit 1. PLI Core Components 
Program Area PLI Component 


The parent resource room is an adult-focused space dedicated to 
providing parents with a welcoming area within the school 


Space in the school 


Parent engagement coordinators (PECs) coordinate closely with other CA 
staff in their schools, school leadership, the local parent association, 
parents, teachers, and the district-assigned parent coordinators or other 
staff designated to work directly with parents in the participating schools. 


Dedicated staff 


PECs: (1) coordinate the design and delivery of adult education workshops 
and parent leadership development activities; (2) help parents link with 
Tailored services and supports needed resources both within the school and within the broader 
community as needed; and (3) work with other school staff to support 
effective parent connections 


PLI Partner Schools 


Each of the PLI partner schools continued to operate as a New York community school during the school 
year with Children’s Aid serving as the lead community school partner for all but one school. Although 
six schools, including the four co-located schools, are included as part of the formal i3 grant, the schools 
are also co-located with three additional unfunded schools which the project director and PECs indicate 
are also included in all provided activities and services to the extent possible. 


Each of the participating schools have relatively small enrollments and, like other Bronx schools, are 
characterized by comparatively high enrollment rates of students with disabilities, economically 
disadvantaged students, and English language learners. Nearly all enrolled students represent a racial or 
ethnic minority group, and between 23 and 62 percent of students were labeled as chronically absent 
(missed 10 percent or more of school days) during the 2016-17 school year. For all but one of the partner 


schools, fewer than 20 percent of students met annual English Language Arts and mathematics 
performance targets as measured on the New York State Department of Education Assessments (Exhibit 2). 


Report Overview 


This report serves as the final report on this phase of the PLI and includes an exploration of 
implementation during year 4 (2016-17 school year) and analyses of quantitative data on student 
academic performance. The next section of the report provides an overview of evaluation goals and 
methods. The following section includes a discussion of Year 4 implementation and impact findings. 
The final section includes summary conclusions and recommendations for continued program 
improvement. 


Evaluation Goals and Methods 


Children’s Aid contracted with Policy Studies Associates in early 2014 to conduct an evaluation of the 
implementation and impact of the PLI in participating schools. The evaluation period included the 2014- 
15 through 2016-17 school years. Throughout the evaluation period, the evaluation explored the 
following proposition: 


Through the coordinated efforts of Parent Resource Centers, parent engagement coordinators, 
and adult education and leadership development activities, parents will develop home 
environments that promote learning and will forge stronger connections with their children’s 
schools, resulting in student achievement and attendance that exceed those of similar students 
enrolled in matched schools. 


Exhibit 2. PLI School Characteristics - SY 2016-17 


A 321 PK-5 99% 32% 15% 85% 34% 5% 6% 
B 420 K-5 98% 20% 5% 79% N/A 61% 61% 
Cc 591 PK-8 99% 23% 25% 88% 26% 19% 18% 
D 293 PK-5 99% 18% 11% 89% 23% 15% 18% 
E 272 6-8 98% 26% 15% 86% 23% 3% 18% 
F 483 9-12 100% 28% 12% 82% 62% N/A N/A 


Exhibit reads: Source: http://schools.nyc.gov/Accountability/tools/report/default.htm 
Note: Percentages are rounded 


*The highlighted schools indicate partner schools which are served by a single PEC and share a resource room. 


Implementation Data Collection and Analysis 


For the 2014-15, 2015-16, and 2016-17 school years, the evaluation team focused evaluation efforts on 
developing an in-depth understanding of what the PLI looks like in practice across each of the six partner 
schools (Hildreth, Butler, & Francis, 2016; Hildreth, Butler, & Orozco, 2017). The primary goal of the 
final, Year 4 implementation evaluation was to document via interviews with representatives of key 
stakeholder groups and analyses of administrative data on parent participation in sponsored services, 
strategies for implementing PLI during the 2016-17 school year. Of particular interest was learning 
about ways that CA staff worked with school staff to integrate parent engagement activities into overall 
school operation. 


In late spring 2016-17 members of the evaluation team conducted individual interviews or focus groups with 
more than 30 stakeholders representing each of the participating partner schools, the New York Department 
of Education, a partner organization, and CA staff. Analyses of these interviews serve as the primary basis 
for this report. To supplement these analyses, the evaluation team also analyzed CA administrative data on 
the types of services and supports provided to parents. These data provide additional context to the 
evaluation and provide information on “fidelity of implementation” as required by i3. 


Data analyses included a review of all interview transcripts to identify themes about PLI implementation 
and impact. To analyze data on program operation, evaluation team members created a database to 
facilitate tracking toward implementation targets. 


Impact Data Collection and Analysis 


The final phase of the evaluation incorporated quantitative analyses of data on student academic 
progress and performance prior to and during PLI implementation. To isolate the potential impact of 
the PLI on student outcomes, the evaluation compared6+the performance of the six PLI schools in 
mathematics, English/language arts, and attendance with that of 18 similar non-participating schools 
New York City public schools. The primary analytic method was an interrupted time-series model which 
looks for shifts or changes in key outcome areas after implementation of a new intervention or program 
compared with trends prior to the change in practice. 


The impact analysis sought to answer the following three questions: 


m At the end of three years of program implementation, what are the average school-level 
proficiency rates on the state reading assessment of students enrolled in grades 3-8 for schools 
that have received PLI supports from CA, compared to the average proficiency rates of similar 
schools that do not receive the CA supports? 


m At the end of three years of program implementation, what are the average school-level 
proficiency rates on the state mathematics assessment of students enrolled in grades 3-8 for 
schools that have received PLI supports from CA, compared to the average proficiency rates of 
similar schools that do not receive the CA supports? 


m At the end of three years of program implementation, what are the average school-level 
attendance rates of students enrolled in grades PK-12 for schools that have received PLI 


supports from CA, compared to the average proficiency rates of similar schools that do not 
receive the CA supports? 


Selection of Comparison Schools 


In early 2014, members of the evaluation identified three comparison schools (18 in total), all located 

within the New York City school district, for each PLI school for the impact analysis. Using propensity 

score matching, the research team identified comparison schools using school-level data based on the 
following criteria: 


Grades served 

Number of students enrolled 

Percent minority 

Percent of economically disadvantaged students, as measured by free and reduced-price meals 
Performance on 2013-14 reading and mathematics assessments 

Percent English language learners 

School climate measures of parent communication and engagement 


All treatment and comparison schools received the “business as usual” parent involvement supports 
provided by the New York City school districts. Since 2003, the NYCDOE has provided schools parent 
coordinators, who are typically responsible for conducting parent workshops, sharing information about 
parent-teacher conferences, and trouble-shooting problems with parents. Both treatment and 
comparison schools had at least one district-assigned parent coordinator. Treatment and comparison 
schools may also have formal parent organizations or formal relationships with local organizations, such 
as ASPIRA of New York, Inc., that support parent engagement. 


Baseline equivalence 


The evaluation team established baseline equivalence of the treatment and comparison schools using 
student-level data from the 2013-14 school year, the year immediately before the initiation of PLI 
activities in treatment schools. PSA fit the following two-level model with grades nested within schools 
(level 1) and schools (level 2). The full model specification can be found in this report’s appendix. For 
the reading and mathematics assessments, PSA researchers standardized student scores; therefore, the 
baseline treatment-comparison difference is in effect size units. These data indicated that there were 
no significant differences in the performance of students enrolled PLI schools and students enrolled in 
comparison schools prior the PLI launch in fall 2014 (Exhibit 3). 


Exhibit 3. Baseline Equivalence Testing 


Treatment- 
Baseline Treatment Comparison Comparison comparison Standardized 
measure group N group N group mean? difference difference 
New York State 
reading 833 355) -0.03 -0.15 -0.15 
assessment 
New York State 
mathematics 851 3,818 -0.10 -0.04 -0.04 
assessment 
student 2,202 7,732 82.82 2.09 0.12 
attendance 


Note: The comparison group mean and treatment-comparison difference are in standardized units. 


Analytic Procedures 


Using data from the 2017 school year, PSA researchers fit a three-level impact model for each of the 
three outcomes (i.e., standardized reading and mathematics scores and student attendance) with 
observations within grades (level 1), grades within schools (level 2), and schools (level 3). At the school 
level, PSA included the following controls: 


m Percent economically disadvantaged (i.e., the proportion of students who received free- and 
reduced-price meals), 

Percent of students from minoritized populations, 

Percent of students who were English language learners 

Number of students enrolled in the school 

School climate scores for parent communication and engagement 

The number of students who took the reading and mathematics assessments 

The number of students who scored proficient on reading and mathematics assessments 
The propensity score for the school 


The full model specification can be found in this report’s appendix. 


1 We standardized students’ ELA and math scores to investigate the impact on students’ scores while accounting 
for the different scoring ranges used across grade levels. The z-score represents the number of standard 
deviations a score is from the mean. To calculate the z-scores, we divided students’ scores by the mean score for 
all students included the sample in the same grade (e.g., a third-grade student’s score was divided by the mean of 
all third-grade students in the sample). We then aggregated the student-level scores to create grade-level z- 
scores. An effect size provides information about the magnitude of an observed effect. In contrast with p values, 
which indicate whether an effect exists, effect sizes indicate the size of the difference between two groups. Effect 
sizes help create a standardized meaning of the difference between groups, regardless of the effect’s scale. 
Researchers use a variety of benchmarks to interpret effect sizes; one approach, proposed by Cohen (1988), refers 
to effects as small (d = 0.2), medium (d = 0.5), and large (d = 0.8). 


Implementation and Impact Findings 


Assessing Fidelity of Implementation 


As required by the i3 grant, staff from PSA and CA developed a logic model (Exhibit 4) detailing the 
planned operation and impact of the PLI. The initial model incorporated the components of the PLI as a 
separate, externally-developed initiative and neglected to account for the potential, and often significant 
impact, of the context of the partner school on PLI implementation and outcomes. Interviews conducted 
during the 2015-16 school year (Year 3) revealed the impact that school context, particularly as it related 
to principal understanding and buy-in, had on the nature of PLI implementation across partner schools. 


In response to these findings, the evaluation team updated the logic model with key contextual factors 
to better reflect the different factors that may affect PLI parent engagement efforts. The updated logic 
model highlights several school context factors which can affect parent engagement efforts in partner 
schools including: (1) connections with the principal and other school staff (2) level of integration with 
other community school services and supports; and (3) development of partnerships with other 
community organizations. Each of these three factors potentially affects the processes of developing 
and maintaining a shared vision of parent engagement in the local school context. The evaluation team 
updated the contextual components of the logic model again based on Year 4 data collection findings to 
include the potential role of local area superintendents and district expectations for the type and nature 
of parent engagement activities across schools. 


Findings from Year 3 and 4 data collection align with prior research on the design and implementation of 
family engagement work. The dual-capacity framework (SEDL, 2013) highlights the important role that 
school context plays in terms of multiple factors including (1) openness of administrative staff to working 
with parents in new ways; (2) school and district policies and procedures dedicated to fostering deep, 
integrated partnerships with parents; and (3) a shared, building-wide commitment to building the capacity 
of both parents and staff to partner successfully. The framework notes that school conditions can foster 
school-family partnerships along a continuum of ineffective to effective partnerships. An ineffective school 
lacks any meaningful opportunities for school staff and parents to build partnership capacity. An effective 
school is characterized by staff who recognize the contributions that parents can make to student learning 
while providing multiple opportunities for family members to engage with their child around learning and 
within the school overall. 


Over the course of the evaluation, the evaluation team has observed the complexity of the work of the PLI. 
As noted above, the PLI is an externally developed initiative which is nested into schools of various levels of 
interest and capacity in building new relationships with parents. Moreover, schools themselves are nested 
within larger administrative regions and the overall New York City school district. Adding to this complexity, 
the PLI is one of many programs and initiatives operated by Children's Aid. Available information suggests 
that this nesting of the PLI in Children’s Aid, the partner school, and regional and districtwide offices affects 
the nature of initiative activities within and across schools. Discussions with CA staff have revealed that CA, 
as an organization, has evolved and continues to evolve as it makes organizational adjustments to develop 
and implement a more broadly defined view of parental involvement in schools. During interviews, CA 
staff discussed different challenges faced in building internal consensus on what effective, meaningful 
parent engagement activities and support looks like in practice. 


Within the i3 evaluation framework, the logic model is directly linked to measures of fidelity of 
implementation. 13 requires that grantee establish targets for key inputs, activities, and outputs as 
outlined in the logic model. Staff from CA and PSA jointly developed implementation targets in Year 1 of 
the PLI and made slight adjustments to the targets after Year 2 to better reflect the realities of on-the- 
ground implementation. Parent engagement coordinators are responsible for collecting information on 
parent participation in PLI services and/or workshops and entering those data into the CA data 
management system. Each year the PLI director provides an export of these data to the evaluation team 
who compile the data to determine CA success in achieving implementation targets. Exhibit 5 provides 
each of the fidelity of implementation targets for the 2016-17 school year. We include the completed 
Year 4 fidelity of implementation matrix in Appendix A-2. 


Year 4 Fidelity of Implementation 


In this section we explore the PLI during 2016-17 in terms of meeting fidelity of implementation targets 
and understanding strategies for working with representatives of the different stakeholder groups. 
Within the framework of the logic model (Exhibit 3), inputs are the different types of resources that an 
organization provides or uses in support of an initiative or program. Program staff and partners, in turn, 
leverage use of these resources to support the design and delivery of activities in support of a program’s 
long and short-term goals (e.g. program implementation). In this section, we discuss each of the key 
measures included in the fidelity of implementation matrix. In the first section we provide a discussion 
of the inputs, activities, and outputs as they operated during the school year. Where applicable, we 
include the extent to which CA was successful in achieving fidelity of implementation targets for the 
school year. 
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Exhibit 4. PLI Logic Model 


Hire, train, place, 
and support 
effective Parent 
Engagement 
Coordinators 
(PEC) 


Parent Resource 
Center 


Conduct adult education & 
training courses & 
workshops (e.g. health and 
wellness, adult education, 
home-school connection; 
community-building, social- 
emotional learning, 
volunteerism and 
leadership) 


Parents participate in 
multiple courses and 
workshops throughout the 
year. 


Parents increase time 
spent helping children 
with home learning 
activities 


Partnership with 
the Schoo 
Principal and 
other school staff 


Integration with 
other CAS 
school-based 
services (e.g. 
community school 
and afterschool) 


Provide resources & 
support through the parent 
resource center to 
strengthen home-school 
connections, including 
providing information and 
support (i.e., translations; 
volunteering opportunities) 


Parents visit school 
resource centers regularly. 


Parents and families make 
connections to community 
resources. 


More parents attend 


parent-teacher 
conferences 


Parents communicate 
with school and CAS 
staff more frequently 


Effective 
partnerships with 
external 
community-based 
organizations and 
services or 
supports 


Regular meetings between 
PEC, principal, and CAS 
director identify parent 
needs and develop tailored 
supportive services; 
provide support to 
teachers, and develop a 
coordinated calendar of 
school events 


Stronger home-school 
connections 


Provide case management 
services and 
service/support referrals to 
serve high need parents, 
families, and targeted 
students as needed 


Contextual Factors: (Hinders or su 


Stronger CAS-school 
partnership 


More parents attend 
school-wide events 
and volunteer at 
school 


Parents attach 
greater importance to 
education and feel 
more confident in the 
school environment 


Increased school 
attendance 


Improved 
classroom 
behavior (e.g. 
reduced 
suspensions)* 


Increased 
student 
educational 
aspirations and 
attachment to 
school 


Improved 
academic 
achievement 


Increased 
progress 
toward high 
school 
graduation 


Prior nature of relationship between schools and parents; readiness of school principal and teachers to accept non-traditional parent involvement and 


support; ites experiences of parents in the school environment. School district and local superintendent messages about and accountability for family 
engagemen 


Exhibit 5. PLI Fidelity of Implementation Measures 


Level Adequate Fidelity at 
Major Component Subcomponents Fidelity Scores Calculated Campus/School level Program-level 


Establishing Parent Resource Allocation of space inthe | Not completed =0 Campus Campus has dedicated 


Centers at Each School school building Completed = 1 space for center = 1 
Hours Center is scheduled | Not scheduled = 0 Campus Campus center has 
to open spans both the Scheduled = 1 scheduled hours during day, 


school day, afternoons, 
| and evenings - 


after school, and evenings = 
1 


engagement coordinators | = trained = 1 

planned for each school Not able to hire 

ls and trained by target number = 0 

The proportion of family Not qualified = 0 Campus At least 90% of staff meet 
engagement coordinators | Qualified = 1 qualifications for their 
with qualifications set for position = 1 

their position 


Offer planned GED, ESL & 


Offered all planned Campus 100% of planned courses 
Technology courses 


courses during the offered = 1 
school year = 1 
Fewer than planned 
courses actually 
offered = 0 


Conduct adult education and 
training courses and workshops 


: Provide Resources/support 
Through Parent Resource Center 


Provide information and Available = 1 
support in advance of Not available = 0 
parent teacher 
conferences 


| Provide services to at least 
10% of parents = 1 


Campus 


? Because two sets of schools share a single school building/campus and PEC, we collapsed these two schools into a single campus and aggregated activities 
into a single campus. Fidelity of implementation analyses include all six PLI schools collapsed into four campuses. 
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Level Adequate Fidelity at 

Major Component Subcomponents Fidelity Scores Calculated Campus/School level Program-level 

Provide help desk, Available = ampus Provide services to at leas 

support center during Not available = 0 10% of parents = 1 

parent teacher 

conferences 

Provide individualized Available = 1 Campus Provide services to at least 

support on Not available = 0 10% of parents = 1 

communicating with 

teachers 

Provide opportunities for | Available = 1 Campus Provide services to at least 

parents to sign up for Not available = 0 20% of parents = 1 

school leadership and 

volunteer activities 

Targeted outreach to Outreach efforts Campus Contact at least 50% of 


Provide parents with 
information about available 
community resources 


Parents participate in courses 
and workshops 


parents of struggling 
students 


Provide information, 
supports, and referrals for 
other parent and family 
needs (e.g., housing and 
emergency assistance; 
sexual health) 


Parents complete at least 
one course or workshop 


Parents achieve 
leadership level (complete 
2 courses and core 
workshops) 


| leadership level = 1 


made =1 
Efforts not made = 0 


Not completed = 0 
Completed = 1 


Parent does not 
complete course or 
workshop = 0 
Parent completes 
course or workshop 


Parent does not 
complete leadership 
level = 0 

Parent completes 


Campus 


‘| 50 parents per school 


parents of struggling 
students = 1 


At least 30 parents per 


or referrals = 1 


complete a course = 1 
< 50 parents = 0 


30 parents per school 
complete training level = 1 
< 30 parents = 0 


campus receive information 


Availability of Parent Resource Rooms 


Implementation Targets: Availability, Met; Hours of availability, Met 


In Year 4, CA continued to meet the 


implementation target for the availability of parent “THE SCHOOL CAN BE LIKE A COMMUNITY 
resource rooms with all four campuses having a HUB FOR PARENTS WHERE THEY’RE GETTING 
dedicated SPareunlOWEVEl as In PRSt years, RESOURCES OR EXPERIENCES THAT HELP TO 
interview participants reported that the ELEVATE THE CHILD’S EXPERIENCES AT 
functionality of the space and the extent to which SCHOOL.” 


it fostered the types of family engagement Children’s Aid Community School 
promoted by the PLI varied across campuses. These Director 


variations reflect the challenges CA has faced since 
Year 1 in securing access to a separate, dedicated 
space in partner schools. Space available for parent resource rooms varied from a small office off the 
school’s main office for one campus to a large, recently renovated space that also began housing the 
school’s food pantry in Year 4. The space for another campus was in the process of renovations at the 
end of the 2016-17 school year. 


During each year of the PLI evaluation, respondents representing school staff, parents, and CA staff have 
highlighted the importance of this space for facilitating increased levels of family and parent 
engagement. During interviews, participants discussed the ways in which the space can help facilitate 
parents’ levels of comfort and frequency of time spent on campus. For example, the principal of the 
school with the small office off the main office noted that he never saw parents congregating in that 
space. The school’s PEC agreed and commented that the size of the room and the proximity to the main 
office likely dissuaded parents from wanting to spend time in the room. This description of the resource 
room stood in contrast to the school that housed a separate space for parents and served as a location 
for many PLI activities including ESL and GED classes. One PEC described the space as being a “neutral 
zone” and a space where parents feel safe and comfortable. A community school director added that 
having a “specialized space” parents aligns with the school’s goal of “meeting families where they are.” 
A CA staff person emphasized the importance of the neutral space and discussed the ways in which staff 
have viewed the space has evolved over the course of the PLI. The staff person stated, that it became 
increasingly important over time to use the space on each campus and to sponsor activities only in ways 
that aligned with parents’ expressed needs. The staff person explained, 


Because, everywhere else in the school, what's happening in there is dictated by the 
principal or the teachers. So being able to really create a space where there can be a 
different power dynamic and real collaboration between parents and school staff, 
parents with each other, regardless of who their kid is at a school; that is becoming 
more and more critical, we see, to the work. 


The staff member added that maintaining the neutrality of this space grew to be a “non-negotiable” or 
core aspect of the PLI model over time. 


A review of calendars for the school year and discussions with PLI stakeholders continue to indicate that 
the centers operated throughout the school year with activities offered at some schools during the day 
and at others on evenings or weekends to best meet the needs of parents. One PEC noted that 


consistently offering services on a regular schedule in the same space has been critical for building 
parent buy-in and trust over time. The coordinator stated, “We have ESL every Tuesday and Thursday, 
no matter what. We don’t move them; this is their room.” Some PECs also noted that teachers and 
other school staff used the rooms for other non-PLI activities. 


PEC Staffing 


Implementation Targets: Number, Met; Appropriate qualifications: Met 


Throughout PLI implementation, CA had notable success in maintaining stability in PEC staffing. During 
the first three years of operation, CA replaced one staff person during Year 2. There was no additional 
turnover in PEC staff during Years 3 and 4. This stability of staffing has likely had a positive effect on PLI 
implementation stability over time and reflects positively on the strategy for hiring and placement that 
was developed during Year 1. Describing the initial hiring process, one CA community school director 
stated, “I think the [hiring] process that we engaged in, we did a fishbowl, it was a very rigorous 
process... and | think that really set the tone in the right direction for us, and it has been huge for us. | 
think for other programs that's probably the most important decision they make.” The program director 
added that there was an intense focus on hiring staff who were familiar with and lived in the local 
community. Interviews with parents, teachers, and partners reveal high levels of satisfaction with the 
PECs with people highlighting the flexibility and commitment of PECs to doing what is necessary to meet 
the needs of parents and to serving as important members of the school community. 


Parents who participated in focus groups during the spring site visits also spoke very highly of their 
contacts with their school’s PECs. Describing the impact of the PEC, one parent stated, “She is the spark 
of the program. When there is a workshop, she is outside yelling, ‘parents, parents, come we have a 
workshop today’.” Similarly, a parent described the work of the PEC saying, “She motivates. She makes 
you feel part of the team, she calls you; she looks for you.” 


Adult Education and Training Courses and Workshops 


Implementation Target: Offer planned courses including GED, ESL, technology 
courses, etc.: Met 


Both interview and administrative data indicate that CA staff successfully delivered a range of 
workshops and activities to families at all partner schools during the 2016-17 school year. Both PLI and 
other school staff describe on-going efforts to tailor services to meet the needs of the families and 
broader community served. PECs reported sponsoring or leading a total of more than 200 different 
activities across the four campuses during the school year. Available programming during the school 
year included regular ESL and GED classes, family archery, healthy cooking, parent organization 
meetings, college enrollment planning, school choice planning and preparation, stress management, 
resume development, and immigration support services. 


The topics of activities varied both across and within campuses and reflected the emphasis placed on 
tailoring services to meet the needs and interests of families. For example, one community school 
director described school efforts to provide services and supports to the school’s grandparents, parents 
who had recently immigrated from Africa, and families which included an incarcerated parent or other 


family member. Other examples of targeted services include separate groups for fathers and mothers 
and for Spanish-speaking parents. A community school director stated, “We will ask them want they 
want, what they need, what they think is important, or we use a little bit of research so we become 
experts on the topics or common issues that affect that group [of parents].” A representative from an 
organization that partners with CA highlighted staff efforts to “listen to the interests and needs of the 
people that they are serving; they’re not coming in with an agenda per se.” The representative 
commented that this strategy helps make family members feel more comfortable in the school building 
and also helps to build a sense of community among parents and other family members. Interviews with 
parents from two campuses confirmed staff reports about efforts to engage parents in the selection and 
design of activities. At the core of this focus is an understanding that parents choose to be involved in 
the school and tailoring services increases the likelihood that parents will make the choice to participate 
in engagement activities. 


In contrast to prior years and a key aspect of this tailoring process, Year 4 planning included regular 
discussions and joint planning with the community school director and most district parent coordinators. 
Staff from all campuses described on-going strategies to ensure regular communication among key 
parties and to plan services in a manner that limited the duplication of efforts and ensured that a wide 
range of services were provided to families. Staff from all campuses discussed regular check-ins among 
staff and focus groups with parents to help shape the types of services and supports provided. A 
community school director reported that jointly working on calendars was an important organizational 
strategy for the school year and helped ensure coordination of efforts. The program director added that 
joint planning also provided opportunities for PLI staff to introduce new ways to engage with parents to 
other school staff. When asked to discuss changes in PLI implementation over time, one community 
school director stated, “I think our capacity is different. Our ability to provide so many opportunities for 
parents including workshops and partnerships.” He also added that the level of coordination with the 
district-funded parent coordinator had improved, and there was less competition as the role of the PECs 
and the PLI became more clearly defined, stating “at this point, it doesn’t feel competitive. It’s just do 
parent work, and we all work together to get it done as opposed to asking, ‘oh where is this coming 
from’?” 


Provide Resources/Support Through Parent Resource Center 


Implementation Target: Offer specified services to 10 percent of parents, Not met 


CA has struggled to meet the implementation indicators associated with the delivery of specific services 
to 10 percent of parents and providing targeted support services for struggling students to 50 percent of 
identified students. One campus met the 10 percent threshold for parent support during conferences 
and support for communication with teachers. None of the other three campuses met any of these 
indicators. In prior years, staff had difficulties consistently entering case management information into 
the data management system. 


Over the course of the three years of operation, Children’s Aid has shifted its focus from case 
management of specific family needs to delivering specific workshops and activities centered around the 
resource room and building stronger connections with other partners both inside and outside of the 
school. The project director noted that Children’s Aid has placed social workers in most schools as part 
of community school wrap-around services, and they are moving toward having social workers focus on 
case management services as needed by families. She noted that during the second year, PECs openly 


advertised access to the New York Times Neediest fund which provides direct funding to families to 
meet emergency needs. By Year 4, although those funds were still available, PL! staff were less likely to 
announce the availability to parents. One school added a food pantry to the resource center and 
provided services to hundreds of families out of the center. 


Provide Parents with Information about Available Community 
Resources 


Implementation Target: Provide information and support to at least 30 parents, 
Not met 


As with the prior indicator, capturing the reach of PLI services continued to be a challenge for Year 4. No 
schools met the implementation target for the 2016-17 school year. 


Parents Participate in Courses and Workshops 


Implementation Target: At least 50 parents complete at least one course or 
workshop per campus, Met. At least 30 parents achieve leadership level 
(complete at least 23 hours of workshops), Met 


Parent engagement coordinators were successful in meeting implementation targets for parent 
participation in workshops during the 2016-17 school year. PECs from all four campuses were successful 
in having at least 50 parents per campus complete one course or workshop. The number of participants 
in workshops continued to increase in Year 4. The number of adult participants increased from 856 in 
Year 2 to 1,887 in Year 3 to 2,817 in Year 4. The number of parents participating in an activity across 
campuses ranged from 327 to 952. 


Additionally, three of four campuses were successful in having at least 30 parents achieve leadership 
status by completing 23 or more hours of courses or workshops. The number of parents achieving 
leadership status increased from 157 to 169 from Year 3 to Year 4. A total of 97 parents achieved 
leadership status in Year 2. Among parents achieving leadership status during the 2016-17 school year, 
approximately 46 percent of parents had previously achieved leadership status during a prior school 
year, and 54 percent had achieved the status for the first time during the year. 


Working with Principals and Teachers 


Year 3 findings highlighted the role that school context played in the ways that the PLI has been 
implemented over time. The framework for understanding the continuum of school contexts outlined at 
beginning of this report provides a useful context for understanding goals and operation of the PLI. The 
development of strong partnerships with community school staff, teachers, principals, parents, and staff 
from partner organizations serves as an important but hard-to-quantify aspect of PLI operation. 
Interviews with PECs, community school directors, district parent engagement coordinators, principals, 
and parents underscore the challenges faced in developing and maintaining these partnerships over time. 
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Working with principals. Principals play critical 
roles in shaping how schools function. Across all 
campuses staff noted that, at some level, 
principals from all schools recognized the 
importance of securing parental involvement 
and the potential positive impact of parental 
involvement on student achievement. A CA staff 
person commented that findings from the Year 
3 report revealed a gap “between our 
understanding of the practices around parent engagement, versus what principals understand and/or 
want.” She added that the principals do not have to engage in the PLI if they choose not to. To address 
this gap the PLI program director focused efforts on modifying how the program interacted with 
principals on two key levels: (1) within each partner school and (2) at the district level by developing an 
understanding of the messages about family engagement that principals received from regional and 
district offices and office staff. The staff person commented that the district-level work was a key shift 
in practice that they had not anticipated when initially designing the program. 


“BUT HERE, | HAVE LOST THE FEAR. AND 
WITH MY SON, | CAN READ TO HIM . | CAN SAY 
THE ALPHABET WITH HIM. AND IN NUTRITION 
CLASS, IT’S GOOD FOR OUR HEALTH TO KNOW 


THESE THINGS, TO KNOW WHAT'S GOOD FOR 
MY SON SOHE CAN STAY HEALTHY.” 
PLI Parent 


Despite these efforts, principal understanding of the full extent and goals of the PLI continued to vary 
across campuses in Year 4. Interviews with nearly all principals revealed a clear tension between a more 
direct focus on parents as facilitators of student academic achievement and engagement and the more 
seemingly indirect focus on increasing parent capacity as promoted by the PLI. Interviews with 
principals also indicate that accountability pressures from the school district to meet specific goals for 
parent engagement may be the source of some of these tensions. These pressures may be especially 
challenging if the PLI operates on a co-located campus because the PEC must balance meeting the needs 
of different principals who might have different goals or visions for PLI operation. 


Principal interviews highlight these tensions. For example, one principal commented that he was not 
fully clear on PLI goals. He stated, “I think one of the things that | would like to know more about [is] 
what Children’s Aid vision for parent engagement is and how that would combine with what our vision 
is.” Principals from two schools emphasized the importance that they placed on having the PEC help 
them meet the parent engagement duties as captured in their annual principal evaluation. One 
principal stated, 


| have requirements that | need to meet. Those academic outcomes that | need to 
meet...The feedback that | get when I'm evaluated against the quality review or my 
performance, from my superintendent. And so | have to put an action plan into place also. 


Another principal expressed a similar sentiment highlighting the tension between an explicit focus on 
serving parents as individuals versus serving parents as means of facilitating increased student 
engagement and school improvement. Discussing strategies to improve PEC hiring by Children’s Aid, the 
principal stated: 


| would say that the organizational weakness for Children's Aid might be what their 
vetting process is to bring in individuals who all have that idea that our job here is to 
support the principal and the school, not thinking of themselves as individuals or as 
separate from the school, but thinking of themselves as part of the school. 


This response highlights a persisting gap between the PLI vision and the principal’s vision for parent 
engagement. The CA vision for parent engagement prioritizes the parent as the recipient of services and 
support and helping parents “set the stage” for student academic success. This focus stands in contrast to 
a vision of engagement which prioritizes the principal 
or students. The ultimate goals of each of the visions TO KNOW THAT YOU CAN COME IN AND 
for parent engagement are the same: enable students IT'S NOT THREATENING. YOU CAN COME IN 
to be more engaged in school and experience 
increased levels of academic achievement. It is the PROJECTS WITH THEM, AND THE 
paths by which these goals are achieved that differs. MATERIALS ARE THERE, AND THERE'S A 
This is not to say that CA staff don't appreciate the RESOURCE AND THERE'S AN OUTLET. EVEN 
importance of being directly supportive of the IE THE PARENT DOES HAVE A STRUGGLE... 
principal and his/her goals for the school. A CA staff IT COULD BE WITH A TEACHER, IT COULD 
person discussed the transactional nature of BE WITH THE SCHOOL, IT COULD BE WITH 
relationships between external and school-based staff WHATEVER, TO KNOW THAT THERE'S A 
in some schools. In these situations, external staff NEUTRAL VOICE THAT'S NOT THERE TO 
such as the community school director may be asked JUDGE, AND THAT IS THERE TO SUPPORT 
to perform some duties or hold activities which fall THEM AND THEIR CHILD. THAT MAKES IT 
outside the scope of their preferred work. However, 
building trusting relationships with principals is critical 
to the initiative's overall success so some level of 
transactional work may have to occur. The CA staff 
person noted that it is important for staff to balance these competing demands. She stated: 


AND SIT WITH YOUR CHILD AND DO 


ALL THE MORE WORTHWHILE. ” 
Teacher in PLI school 


So we've had to really work with the [community school] directors. Both kind of 
understanding the position they're in, supporting them because they do a lot of work 
to clear the path for the parent engagement work; while at the same time, figuring out 
those points where, no, this is a non-negotiable point in terms of the practice, because 
it might undermine the parent's trust in what's happening there, or it might send the 
wrong message, which would undermine trust. Those things we've had to do a little bit 
more work around. That's not in every place, but in some places, where it tends to 
show up more. 


Discussing the different roles between the work of the PEC and the district coordinator in her school, 
another principal stated that she prioritized the public relations role of the parent engagement staff 
person, stating that a key role for the school’s district-funded parent coordinator was to “get our 
message out there...to be the first face our parents see when they come to the school.” The principal 
also noted that the district coordinator also provided a level of “interference” when a parent just 
needed to talk. 


Reflecting on this aspect of school context, a CA staff person emphasized the importance of PECs 
maintaining neutrality between parents and the principal when operating in the school rather than 
being seen by parents as an extension of the principal. The staff person noted that gaining the trust of 
parents is critical for the type of work called for by the PLI. The almost inevitable dynamics of power 
that come to play in schools when dealing with the principal has the potential of undermining this trust. 
Discussing the relative independence that the PECs have from the school’s principal compared with 
district-provided parent coordinators, the CA staff person stated: 


What they've begun to understand is that there are some things that just have to come 
from the school's parent coordinator, versus things that happen in the parent 
engagement center. Ideally, what we'd love to see is that everybody's practice is the 
same, and that tension between parent engagement vis-a-vis the principal in the 
principal's role, versus just good engagement that really reinforces the role of the parent 
in the school community in their child's education; that those things will become one and 
the same. But right now, that's not the case. The parent engagement coordinator really 
being focused on just that, not intervening in a conflict between a principal and a parent. 
In that instance, the power dynamic is all leaned on the principal's end because the 
parent coordinator is employed by who? 


Few PECs described extensive relationships with principals around PLI planning and implementation. 
Both the PECs and the principals stated that principals were more likely to have regular conversations 
with the community school director or the district-funded parent engagement coordinator about parent 
engagement work. Describing the principal’s involvement with family engagement activities, one PEC 
acknowledged the different pressures the principal faced as leader of the school and how those 
pressures might affect the level of engagement with parent activities. She stated, 


| guess that the weight of having the city behind her, teachers, and the daily things that 
happen...| think it’s taken a toll on her, as on all of us, but I feel this is a team effort, and | 
feel that if she could delegate more and have less on her shoulders, it [the PLI] could work 
better. Her role could be more substantial, it could have more meaning, or she could 
reach out to better grasp what she’s supposed to do as principal. 


The PEC indicated that the principal was somewhat separated from the parents and that things might be 
better if she was “out” there more. A PEC who leads activities on a co-located campus described a more 
open relationship with one principal and more challenging relationship with the other principal, stating: 


The principal trusts us so much. We’re able to say, ‘that might not work, let’s try it this 
way.’ Or we’re able to say, ‘we failed, how can we do it better next time?’ We have a 
healthy partnership. We don’t have that relationship [with the other principal]. [The 
principal] wants everything her way. If | produce good work, that’s good for her. We did 
a workshop, she came in and said, ‘This is great. This is what | want to see more. And she 
left. It’s not the best partnership, but at the end of the day...as long as parents feel safe 
in this room and the workshops we do are successful, that’s all that matters. 


PECs from the other two campuses described similar respectful, but somewhat limited, relationships 
with principals. 


Working with teachers. Both PECs and the project director noted increased levels of involvement with 
teachers in partner schools during the 2016-17 school year. All four PECs described more explicit efforts 
to connect with teachers. Teachers who participated in focus groups also reported increased outreach 
by PECs during the school year compared with prior years. Strategies for connecting with teachers 
included reaching out to them individually to see if 
there were ways that the PECs could help teachers 


make better connections with parents and WE'RE PUSHING TEACHERS. WE'RE 
responding to specific requests for assistance [WORKING] NOT ONLY WITH THE 

posed by some teachers. PECs also reported PARENTS, BUT THE TEACHERS AND HOW 
inviting teachers into the resource room so that TO COMMUNICATE WITH PARENTS. AND | 
they could see both the number of parents who THINK, AS TIME GOES BY, THEY SEE THAT 
were in attendance and the types of activities in THERE ARE DIFFERENT WAYS OF 

which they were engaged. One parent engagement COMMUNICATING WITH PARENTS. THERE 
coordinator linked the development of these ARE DIFFERENT WAYS OF ENGAGING A 
relationships with the consistency of services and PARENT. IT'S NOT JUST PARENT/TEACHER 
visibility that the PLI had achieved in the school CONFERENCE. 

over the prior two school years. Over time, PECs 

said that they were able to build credibility with PLI Parent Engagement Coordinator 


and gain the trust of some teachers in the school. 
Describing work with teachers, one PEC stated that she approaches teachers individually to discuss 
parent engagement strategies and the ways that engagement can support student goals for students. 
PECs, teachers, and principals from several of the campuses discussed PEC support of the academic 
parent-teacher teams (APTT) conferences implemented by some PLI schools. These conferences provide 
a platform for the development of a closer partnership between parents and teachers. One PEC also 
discussed working with teachers to offer grade-level mathematics and literacy sessions for parents. 


Teachers from three campuses discussed their work with their school’s PEC. Describing the role and 
impact of the PLI, one teacher stated: 


| see that as a way to engage parents and bring them into the building. If you think 
about a child's life cycle as a triangle in school, one angle is the parent, one angle is the 
child, and one angle is the teacher. And they have to work in conjunction, or else you'll 
have that open shape, and information goes in and out, but it isn’t retained. By having 
your parents actively involved, it creates that support. One good thing about a triangle is 
that you turn it on any side, it will stand. So that parent is very necessary to help that 
triangle remain a structure, its integrity and to stand. | think by these programs... The 
parents feel involved, and they feel that they are important. 


Although some teachers were open to PEC support, PECs noted that not all teachers were open to PLI 
efforts. However, despite some resistance, PECs commented that they continued to attempt to make 
these connections finding that some resistant teachers become less resistant over time, especially if 
they were able to see the partnerships with other teachers in practice. PECs also noted that teacher 
turnover also makes it important for PECs to continue to reach out since new relationships are 
constantly needed to be forged with incoming teachers. 


Working with district staff. CA staff made some efforts to explain its vision of parent engagement with 
staff from the New York Department of Education. The program director reported that the primary goal 
of the district outreach was to increase awareness of the nature of PLI engagement efforts. The project 
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director stated that as part of these efforts she presented to district parent coordinators approximately 
three times about core elements of the PLI approach. Topics of training included: the use of space, 
cultural competence, parent trust, and parent advocacy. A former district staff person praised CA 
efforts to bring a coherent view of parent engagement to the school district. The former staff person 
noted that there has traditionally been no clear vision for parent engagement in the district beyond the 
desire for increased levels of parent involvement. She added that the lack of clarity resulted in a lack of 
consistency of efforts across schools and parent coordinators. 


A Focus on Attendance 


The project director noted that at each school and throughout the evaluation period, PECs supported 
several initiatives to boost student attendance. For example, the PLI supported Attendance Awareness 
Month, during which PECs provided coffee to parents at morning drop-off and discussed the importance 
of consistent student attendance. Parents and their children also participated in a “School Every Day” 
march to celebrate and affirm the importance of attending school. For PLI staff, a key strategy to 
support attendance was communicating the instruction time lost when students are chronically absent. 
As one staff member documented, “Parents were surprised how much their child has actually missed. 
Ten days [out] from school does not sound too bad for some parents, but 60 hours of instruction means 
much more for most.” In addition to direct strategies to support attendance, the PLI’s structure and 
activities indirectly supported student attendance. PLI activities helped parents strengthen their 
networks which, as one staff member explained, helped parents see that they were not alone in 
supporting their child’s development. Similarly, activities centered on parents’ and caregivers’ health 
may increase students’ attendance. One Children’s staff member hypothesized that the PLI-supported 
grandparents’ group may contribute to positive student outcomes: 


[The grandparents’ group] is affecting people’s health because [the grandparents] talk about 
how they’re taking care of themselves and if they’re getting enough rest. Then, you notice the 
kids that were late or chronically absent are going to school more and arriving on time because 
the grandparent has a little more energy and connection to the community. 


In the next section of the report, we focus in impact analyses. These analyses indicate that these 
focused CA efforts have the potential of supporting improved attendance by students. 


Impact Findings 


Our analysis of attendance and achievement outcomes found no statistically significant impacts of the 
Children’s Aid Parent Leadership Institute on students’ math, reading, or attendance. In 2017, the 
comparison group mean attendance was approximately 86 percent. Students in schools implementing 
the PLI had a mean attendance rate that was, on average, 3.5 points higher than their peers in 
comparison schools. However, this difference is not statistically significant. 


For analyses of academic achievement, we standardized students’ scores to create a comparable 


measure across grade levels. In 2017, students in comparison schools had reading scores that were 0.06 
standard deviations above the mean, while students who attended PLI schools had reading scores that 
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were 0.13 standard deviations lower than students in comparison schools. Similarly, in math, students in 
comparison schools had assessment scores that were 0.07 standard deviations above the mean. 
Students who attended schools in PLI schools had scores 0.23 standard deviations lower than students in 
comparison schools. None of the differences between students in treatment and comparison schools 
were statistically significant (Exhibit 6). Appendix A3 contains information on attrition. 


Exhibit 6. Impact Analysis Results 


Impact 


Treatment group Comparison group Comparison group estimate 
Baseline measure N N mean 


New York State 


; 894 2,991 0.06 -0.13 
reading assessment 
New York State 
mathematics 943 3,022 0.07 -0.23 
assessment 
Student attendance 2,001 5,931 85.54 3.54 


Note: The comparison group mean and the impact estimate for ELA and mathematics scores are in standardized units. 


Additional analyses explored whether school-level analyses might provide some indication of differential 
outcomes between PLI and comparison schools. Although the differences in mean student attendance 
rate did not reach statistical significance, the implementation of attendance-focused activities by PECs 
and early indications of improved student attendance have the potential of highlighting an area of 
impact. The PLI continued to operate for two school years beyond the evaluation period. Analyses 
which incorporate these additional two years may yield statistically significant results that build on the 
PLI’s current progress (Exhibit 7). 
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Exhibit 7. Average Attendance, ELA, and Math Outcomes, 2017, by Matched Group 


PLI schools NW eLwel ake Mexednaloy-lakxey amy svete) fy 


Math rN awevave yale Math rNideyarerelares 


Children’s Aid 
College Prep Charter - - = = = ea 


School 

P.S. 211 -0.61 -0.46 89% 0.29 0.62 88% 
PS. 61 -0.33 -0.17 89% 0.14 OZ 88% 
P.S. 314 -1.1 -1.20 88% 0.12 0.27 84% 
Fannie Lou Hamer -0.24 -0.75 90% -0.02 -0.35 88% 
Middle School 

Fannie Lou Hamer High a 71% = = 70% 
School 


Note: ELA and Math scores have been standardized. 
Due to data quality issues, Children’s Aid College Prep Charter School and its matched comparison schools were excluded from 
follow-up analyses. 


Conclusions and Recommendations 


In this section of the report we summarize key findings from Year 4 and the impact analyses and 
highlight potential implications of these findings for future scale-up efforts for Children’s Aid and the PLI. 


Fidelity of Implementation and Potential Changes to the PLI Model 


The PLI continued to operate as initially planned for most key program components. The PLI was 
successful in meeting implementation targets for (1) PEC staffing, (2) the availability of a resource room, 
(3) the delivery of a diverse array of activities and workshops for parents, and (4) the number of parents 
participating in activities and workshops and achieving leadership status. Participation and leadership 
level numbers increased annually between Years 2 and 4. 


As for the other two indicators which reflect the case management aspects of the initial PLI model, some 
campuses were able to achieve one or two of the individual indicators, but the initiative as a whole was 
unable to achieve the overall indicators for any school year. This status likely reflects both the difficulty 
of adequately capturing the more informal aspects of PEC work and the increased focus, over time, on 
working with school staff more closely and developing and delivering a wide range of tailored activities 
for parents as a whole. 
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This shift in focus from individual case management to larger scale activities for parents and more direct 
contact with other school stakeholders potentially has implications for the program model. If this is the 
direction that staff from Children’s Aid advocate, then removing case management activities as indicators 
of PLI operation may be something to consider as the organization reviews ideas for scale-up and 
replication. It is likely that some level of this work will continue as a result of PEC relationships with 
parents but removal of this work as a key area of focus for PECs may be warranted. Continuing to build 
strong connections with the community school director, school social worker, and other school-based staff 
will allow parents to continue to receive needed services and supports while allowing PECs to continue 
focus on joint planning and developing tailored activities. 


Another potential change to the PLI model centers around core components which currently include 
space, staff, and the tailoring of services to parents. However, undergirding the PLI model both explicitly 
and implicitly is the work needed to develop and maintain partnerships with key stakeholders including 
the principal, teachers, and other community school staff. As CA considers options for scaling up and/or 
replicating the PLI model, it may be helpful to consider making this core work more explicitly a component 
of the PLI model. A more formal recognition of the role of these partnerships in the PLI model highlights 
the importance of these partnerships and may encourage earlier conversations with the different 
stakeholder groups about the role the PLI can play in overall school operation. 


Working with Teachers and Principals 


The development of effective partnerships with school staff continued to be an area in which most PECs 
expressed at least some level of challenge. Year 4 saw an increased effort to work more directly with 
teachers around their work with parents. Both PECs and teachers discussed the increased efforts toward 
collaboration. The on-going challenges in developing effective partnerships are likely a reflection of the 
persistent gap between how principals typically view parental involvement and the vision of the PLI. 
More traditional views of parent engagement conceive of parents primarily as monitors and motivators 
of their children. This monitoring and motivating of students may lead to increased levels of student 
performance. The PLI also recognizes the importance of this level of parent involvement. However, the 
focus of the PLI is on increasing the capacity of parents to be effective in these roles and making them 
feel comfortable and welcome in the school via participation in sponsored workshops and activities. In 
the model, increasing parent capacity and level of comfort in the school may result in higher levels of 
student performance by helping parents set the stage for higher levels of student performance or 
engagement. Interview data from Year 4 indicate that more work with principals is likely necessary to 
help craft more closely aligned visions of parent engagement. It will be important for CA staff to clearly 
explain its focus on stage setting versus traditional parent engagement at the beginning of activities and 
making efforts to secure principal understanding and buy in. 


However, despite these gaps in visions for parent engagement, the PLI continued to operate, with 
success, on all campuses during the 2016-17 school year. During interviews, all principals noted that 
they valued parental involvement at some level and also valued the partnership with Children’s Aid. 
These factors appear to provide some level of space for the PLI to operate and provide opportunities for 
parent engagement work on all partner campuses. Referring back to the SEDL continuum of family- 
school partnership, no PLI schools were operating at the completely ineffective level. The increased 
level of partnership with other members of the school community, particularly the community school 
director, allowed the breadth and depth of PLI implementation to increase in Year 4. However, to move 
to fully effective partnerships as defined in the SEDL model, full principal buy-in to the PLI model would 
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be needed. CA may want to consider the extent to which this level of full principal buy-in, as opposed to 
minimal to moderate levels of buy-in where principals are open to but not fully engaged in the work, are 
desired in discussions of scale-up and replication. Data from Years 3 and 4 suggest that the PLI can 
operate with success in schools characterized by a low/moderate level of principal buy-in. Full 
integration of the PLI into overall school operation will, however, only occur with higher levels of 
principal buy-in and engagement. 


Student Impact 


This evaluation suggests the potential for parent-centered engagement models for helping parents set 
the stage for the children’s academic success. The PLI’s program components—family engagement 
centers, dedicated staff, parent workshops, and connections to services—helped parents feel welcome 
in their school community, build and strengthen relationships with other parents, and develop skills to 
support their education and development. Although our impact analysis did not find statistically 
significant effects on students’ educational outcomes, the evaluation’s findings suggest a potential for 
parent-centered programs to reduce chronic absenteeism among students. The PLI’s direct messaging 
about the importance of school attendance may have influenced parents’ efforts to ensure that their 
children attended school as often as possible. Further, as program staff hypothesized, the PLI may have 
indirectly supported improved attendance by helping parents and other caregivers improve their health 
and strengthen their connections with other parents. Additional investigation and analysis may be able 
to establish causal links between the PLI’s program components and improved attendance at school. 


In addition to investigating the impact of the program on student attendance, additional analyses may 
identify the pathways from parent-centered engagement models to effects on student achievement. 
While workshops explicitly focused on academics were not a core component of the model, PECs 
increased their coordination with school staff as the project progressed to plan programming to support 
academics. However, the PLI does require principals and educators to expand their conceptions of school- 
based parent engagement activities in ways that incorporate programming designed to help parents 
improve themselves and foster a stable home environment. Given the program’s primary focus on 
parents, observable effects on students’ academic achievement may be detectable after the study period. 
In the shorter term, the skills, knowledge, and networks parents gained from the PLI may create a critical 
foundation for student achievement and engagement in school throughout their academic careers. 
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Appendix A 


I-V 


Exhibit Al: Baseline Equivalence Testing 


Major Component : Campus B Campus D F 
Possible : : Programmatic 
cae Sub-Components Campus A (includes two Campus C (includes two indicator Met 
schools) schools) 
1.0 Establish Parent 1.1 
Resource Centers at 1 Allocation of space in the school Y Y Y Y Yes 
Each School building 
1.2 
1 Hours Center is scheduled to open y y y y vee 
spans both the school day, 
afternoons, and evenings 
Measures 4 1.0 Total 2 2 2 2 Yes 
2.0 Hire and Train 2.1. The number of family 
parent engagement 1 engagement coordinators planned y y y y an 
coordinators for each school are hired and 
trained by CAS 
2.2 The proportion of family 
1 engagement coordinators with Y Y Y Y Yes 
qualifications set for their position 
Measures 2 2.0 Total 2 2 2 2 Yes 
3.0 Conduct adult 
education and training 1 3.1 Offer planned GED, ESL & y y y y Vee 
courses and Technology courses 
workshops 
Measures 1 3.0 Total Y Y Y Y Yes 
4.0 Provide 4.1 Provide information and 
Resources/support 1 support in advance of parent N N N N No 
Through Parent teacher conferences to 10% of 
Resource Center parents 
4.2 Provide help desk, support 
1 center during parent teacher N Y N N No 


conferences to 10% of parents 


c-V 


Major Component 


5.0 Provide parents 
with information 
about available 
community resources 


6.0 Parents participate 
in courses and 
workshops 


Possible Campus Campus B Programmatic 
Sub-Components Campus A (includes two Campus C (includes two ; 
Score Indicator Met 
schools) schools) 
4.3 Provide individualized support 
1 on communicating with teachers to N Y N N No 
10% of parents 
4.4 Provide opportunities for 
1 parents to sign up for school 7 N N N N No 
leadership and volunteer activities 
to 10% of parents 
4.5 Targeted outreach to parents of 
1 struggling students to 50% of N N N N No 


parents identified 


5.1 Provide information, supports, 
and referrals for other parent and 
family needs (e.g., housing and 
emergency assistance; sexual 
health; at least 30 parents) 


6.1 Parents complete at least one 


1 Y Y Y Nj Yes 
course or workshop 
6.2 Parents achieve leadership level 

1 (complete 23+ hours of workshops; Y Y N Y Yes 


at least 30 parents) 


A2: Baseline equivalence model 


Level-1 Model: Grade Level 

Bogi = Mo + VGj 

Level-2 Model: School Level 

T oj = Yoo + V o1(Tj) + Hoj 

Y o1represents the difference between the baseline mean scores of treatment and comparison schools. 
Exhibit XX displays the results of the baseline equivalence testing. 


Impact analysis model 

Level-1 Model: Student Level 

Zicj= 6 ogj + € ici 

Level-2 Model: Grade Level 

6 ogj = oj + Fj 

6 16j = 74; 

Level-3 Model: School Level 

TMoy= Yoo Vor(Tj) + Yo2(PctDisAdv;) + yos(Minority;) + yo4(ELLj) + yos(Enroll;) + yos(Climate;) +7 
o7NumTestedg;) + Yoa(PctProgj) + Yo9(Propensity;) + Lo; 

y= V10 + 11(T)) 


where, Zig; is the z-score from the i™ time point in the G‘ grade in the j" school, as described above. The 
random term (rg) is assumed distributed normal with mean zero and variance v. The covariates 
“NumTested” and “PctPro” described below are grade-specific and will enter the model as grade-specific 
values. The measures below are school-level measure (not time varying) and represent measures of the 
schools during the pre-treatment years. 
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